Explicit coding in the brain: data-driven semantic analysis of human fMRI BOLD responses with Formal Concept Analysis

摘要

Understanding how semantic information is represented in the brain has been an important research focus of neuroscience in the past few years. We showed previously (Endres et al 2010) that Formal Concept Analysis (FCA, (Ganter and Wille 1999)) can reveal interpretable semantic information (e.g. specialization hierarchies, or feature-based representation) from electrophysiological data. Unlike other analysis methods (e.g. hierarchical clustering), FCA does not impose inappropriate structure on the data. FCA is a mathematical formalism compatible with the explicit coding hypothesis (Foldiak, 2009) Here, we investigate whether similar findings can be obtained from fMRI BOLD responses recorded from human subjects. While the BOLD response provides only an indirect measure of neural activity on a much coarser spatio-temporal scale than electrophysiological recordings, it has the advantage that it can be recorded from humans, which can be questioned about their perceptions during the experiment, thereby obviating the need of interpreting animal behavioural responses. Furthermore, the BOLD signal can be recorded from the whole brain simultaneously. In our experiment, a single human subject was scanned while viewing 72 grayscale pictures of animate and inanimate objects in a target detection task (Siemens Trio 3T scanner, GE-EPI, TE=40ms, 38 axial slices, TR=3.08s, 48 sessions, amounting to a total of 10,176 volume images). These pictures comprise the formal objects for FCA. We computed formal attributes by learning a hierarchical Bayesian classifier, which maps BOLD responses onto binary features, and these features onto object labels. The connectivity matrix between the binary features and the object labels can then serve as the formal context. In line with previous reports, FCA revealed a clear dissociation between animate and inanimate objects in a high-level visual area (inferior temporal cortex, IT), with the inanimate category including plants. The inanimate category was subdivided into plants and non-plants when we increased the number of attributes extracted from the fMRI responses. FCA also highlighted organizational differences between the IT and the primary visual cortex, V1. We show that subjective familiarity and similarity ratings are strongly correlated with the attribute structure computed from the fMRI signal.